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a graduate student in the Department of Computer Science, and Mr. Kaize Adams is a graduate 
student in the Department of Industrial & Systems Engineering. For all the students from North 
Carolina A&T State University who served as non-pilots, I say thank you all. 



ABSTRACT 


The present study investigated effects of weather related factors on the performance of 
pilots under free flight. A weather scenario was defined by a combination of precipitation factors 
(light rain, moderate rain, and heavy rain or snow), visibility (1,4,8 miles), wind conditions 
(light, medium, or heavy), cloud ceiling (800ft. below, 1800ft above, and 4000ft horizontal). The 
performance of the aircraft self-separation was evaluated in terms of detection accuracy and 
detection times for student- and commercial (expert) pilots. Overall, the results obtained from a 
behavioral analysis showed that in general, the ability to recognize intruder aircraft conflict 
incidents, followed by the ability to acquire the spatial location of the intruder aircraft relative to 
ownership aircraft were judged to be the major cognitive tasks as perceived by the participants 
during self-separation. Further, the participants rarely used cockpit display of traffic information 
(CDTI) during conflict management related to aircraft separation, but used CDTI highly during 
decision-making tasks. In all weather scenarios, there were remarkable differences between 
expert and student pilots in detection times. In summary, weather scenarios were observed to 
affect intruder aircraft detection performance accuracies. There was interaction effects between 
weather Scenario- 1 and Scenario-2 for climbing task data generated by both expert- and student- 
pilots at high traffic density. Scenario-3 weather condition provided an opportunity for poor 
detection accuracy as well as detection time increase. This may be attributed to low visibility. 
The intruder aircraft detection times were not affected by the weather conditions during climbing 
and descending tasks. The decision of pilots to fly into certain weather condition was dependent 
in part on the warning distance to the location of the weather. When pilots were warned of the 
weather conditions, they were more likely to fly their aircraft into it, but mostly when the 
warning was not close to the weather location. 
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1. INTRODUCTION 

1.1. Background 

According to the Federal Aviation Administration (FAA) “the annual air traffic 
rate is expected to grow by 3 to 5 percent for at least the next 15 years, and the current 
airspace architecture and management will not be able to efficiently handle this increase 
(http://asd. 0 rlab.faa.g 0 v/files//ff_ 0 v.htm, 6/1 1/98). This increase places a huge load on 
the air traffic controllers. Consequently changes need to be made to the current air 
system. As a result, one solution that has been designed to accommodate the 
overabundance of air traffic is Free Flight. 

Free Flight strives to move the current air traffic system in to an age where space 
technology is used to its fullest potential. Free Flight can be defined as “a safe and 
efficient flight operating capability under instrument flight rules (IFR) in which operators 
have the freedom to select their path and speed in real time” 

(http://www.businesswire.com/emk/1201-9.tx, 6/1 1/98). Free Flight only limits pilots 
freedom in four general cases. Those cases are “to ensure separation, to preclude 
exceeding airport capacity, to prohibit unauthorized flight through special use airspace, 
and to ensure safety” (RTCA, 1995). Because Free Flight is designed to empower pilots 
with new responsibilities, it may relieve the air traffic controllers from the workload that 
is predicted to drastically increase in the near future. 

The FAA’s highest priority operational outcome is to improve safety. The FAA 
has defined a set of safety standards for spacing between multiple aircraft, aircraft and 
other physical structures, and aircraft and airspace. System safety from air traffic 
standpoint, is measured through the ability to maintain these standards. When aircraft 
violate these separation standards, an operational error occurs. Specifically, an 
operational error can occur when (a) Less than the applicable separation minimum results 
between two or more aircraft, or between an aircraft and terrain or obstacles, or (b) An 
aircraft lands or departs on a runway closed to aircraft operating after receiving Air 
Traffic Controller (ATC) authorization. 

1.2. Aircraft Self-Separation 

Self-separation is one part of the Free Flight concept. Self-separation provides pilots 
the opportunity to choose their own route to reach a specified destination provided that 
they maintain the minimum required separation distance between airplanes. The 
sufficient amount of separation distance between airplanes is 5 nautical miles laterally 
and 1000-2000 feet vertically in domestic enroute environment. Two airspace zones are 
designated to monitor this separation: protected zones and alert zones. The protected 
zone is the smaller zone of the two and the one closest to the ownship (the ownship refers 
to the aircraft in questioning). Under no circumstances can this zone be violated (no two 
protected zones can ever touch). It is based on distance, having a radius of one half the 
minimal horizontal separation required (2.5 nautical miles) and ± one half the minimal 
vertical separation required (± 500 or 1000 feet) (RTCA, 1995). The outer or larger zone 
is the alert zone. Unlike the protected zone, the aircraft’s speed and performance 
determines its size. This zone is used to decide if intervention from the air traffic 
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controller is necessary. It is based on a defined time window. “For a given look ahead 
time, the Alert Zone is the locus of all possible Protected Zones of the aircraft at a given 
time ” (RTCA, 1995). This means that the Alert Zone makes the ownship aware of how 
close, with respect to time, it is intruding into the Protected Zone of any surrounding 
aircraft. The alert zone is also said to require the inclusion of human factors’ parameters, 
such as detection time and decision time. Aircraft are allowed to freely maneuver until its 
alert zone touches another aircraft’s alert zone. 

In the event that aircraft alert zones do touch, air traffic controllers have the option of 
intervening in the situation to help the pilots maintain separation. This point is known as 
procedural intervention (Paielli & Erzberger, 1997; Palmer, Jago, & Dubord, 1980). 

Some cases in which procedural intervention may occur are: (a) the workload of the crew 
has become too overwhelming; (b) there is vital information that is known only to the 
controller and not to the pilots, or (c) the controller is uncertain of the decisions that the 
crew is making to resolve a conflict. 

Several tools are already in place that will allow the concept of Free Flight to be used 
as early as today (Johnson, batiste, & Bochow, 1999; Kreifeldt, 1980). The Traffic 
Conflict and Alert System (TCAS), in its advanced state, and related cockpit displays of 
traffic information (CDTI) can be used to help operators maintain self-separation. Global 
Positioning System (GPS) may also be used to determine more accurate locations of 
surrounding aircraft. When GPS is used in conjunction with ADS-B (Automatic 
Dependent Surveillance- Broadcast), location of aircraft can be achieved more rapidly 
(Zeitlin, Hammer, Cieplak, and Olmos, 1998). It is also important to know that only 
certain aircraft may be used for self-separation in the near term. There may be several 
requirements for display characteristics (e.g. display size) that may prevent the use of 
CDTI technology for self-separation in the near-term (Palmer, jago, Baity, & O’Connor, 
1980). 

1.3. Weather Phenomenon 

Weather related accidents are major problems in aviation safety. For example, 
studies in Controlled Flight Into Terrain (CFIT) have consistently mention weather as the 
main factor (Bud, Hannon, Mengert, Ramsom, & Steams, 1997; Driskill, et al., 1997). 
CFIT accidents occur when an aircraft, under complete control of the pilot and crew, is 
unintentionally flown into the ground, with the majority of the incidents occurring during 
low visibility (Wickens, Helleberg, & Xu, 1999). Clearly, loss of situation awareness and 
poor perceptual control of intended actions in the terrain is a major factor arising from 
weather phenomenon (Cashion & Lozito, 1999; Lozito, et al., 1997; O Hare & 
Smitheram,1995). 

Weather conditions can affect the capacity of a destination airport and pilot 
performance in several ways (Pritchett & Hansman, 1997; Wiggins, Martinussen, & 
Hunter, 1999). Low visibility due to clouds or fog can limit the ability of arriving pilots 
to see other aircraft and to see the runway environment. Thunderstorms can greatly 
reduce or stop arrivals to an airport, since aircraft cannot safely fly near thunderstorms. 
Snow or ice on the runway surface can also increase spacing that is required between 
arriving aircraft because of the reduced effectiveness of aircraft brakes and longer landing 
roll. 
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Previous research in free flight environment have investigated the effect of traffic 
density in aircraft self-separation with emphasis on convergence angles in traffic conflicts 
(Lozito, McGann, Mackintosh, & Cashion, 1997; Mackintosh, et al., 1998, Castano & 
Parasuraman, 1999). The impact of weather conditions on free flight needs to be 
examined. This is the major thrust of this research. 

Recently, Wiegmenn, Goh, & O’Hare (2002) have investigated the role of 
situation awareness (SA) on pilots’ decision to fly into adverse weather. Results revealed 
pilots who receive pre-warning on weather will continue to fly into adverse weather if the 
information was received far away from the location of the weather condition. Peterson & 
Uhlarik (1999) and Kreifdelt (1980) observed that even with CDTI supports, pilots are 
likely to be using distance as a cue for making decision about whether to fly into weather 
of change a course of action from original flight plan. A study by Sharma, Pfister, & 
Heath (1999) show how perception of risk by pilots influences their decision to fly 
aircraft into adverse weather. The study indicated the reluctant by pilots to use 
automation when deciding to change flight plans due to incremental weather. Those 
pilots who changed their flight plan do so if the perceive aircraft separation very tight. 

Depending on the visibility conditions, the pilot can either use visual flying rules 
(VFRs) or instrument flying rules (IFRs). VFRs are rules that govern the procedures for 
conducting flight under visual meteorological conditions (VMC). Requirements for visual 
conditions are normally 3-5 miles of visibility and a 1000 foot cloud ceiling. IFRs are 
rules that govern the procedures for conducting flight under instrument meteorological 
conditions (IMC). IMC are meteorological conditions defined by visibility, distance from 
the clouds, and ceiling less than the minima specified for VMC. Pilots operating in IMC 
must comply with IFR, which require the filing of a flight plan, and ATC normally 
provides air traffic separation directives. Pilots may operate under IFR when flying in 
VMC condition. However, ATC will separate only those aircraft complying with IFR. 
Normally, VFR aircrafts provide their own separation, and IFR aircraft have the 
responsibility to see and avoid VFR aircraft (RTCA, 1995).The primary responsibility of 
air traffic controllers is to ensure that a safe separation distance is maintained between all 
IFR aircraft under their control (Krozel & Peters, 1997). 

Goh & Wiegmann (2001a, 2001b) noted that VFR flight into IMC is often 
characterized by the pilot’s decision to continue a flight into adverse weather conditions 
despite warnings from ATC. This behavior is termed a plan continuation event (PCE) by 
Orasanu, Martin, & Davison (2001). PCE related fatalities have been documented in 
aviation studies (Burian, Orasanu, & Hitt, 2000; Goh & Wiegmann, 2001b; McCoy & 
Mikumas, 2000; NTSB,1989; O’Hare & Smitheram, 1995). 

1.4. Project Objective and Scope 

Although aircraft self-separation may prove to be a logical solution to some of the 
problems in the current airways operation, there still may be instances when pilots have 
difficulty with this task (Cashion, et al, 1997). Pilots may find their aircraft in a 
threatening position. They may be intruding into the airspace of surrounding aircraft, 
thereby not maintaining the sufficient amount of separation distance between airplanes. 
One of the problems associated with this situation is the effect of weather on the pilot’s 
flight planning and envisioning process. The major objective of this research is to 
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investigate the pilot performance when in a free flight environment with simulated 

weather conditions. The problems examined and analyzed are: 

1. Aircraft separation procedure: What will pilots do when they need to separate their aircraft 
from traffic and weather events. 

2. Functionality of aircraft separation: Will free flight self-separation be practical with weather 
events in the airspace? 

3. Separation risk: Will pilots take more risks flying into the weather under free flight decision 
making? 

4. Usability of cockpit display aid: Will pilots use cockpit display traffic information (CDTI) 
features differently if they need to consider weather and traffic? 

5. Automation utility: How useful are the conflict probes given that they do not consider 
weather events in the algorithms (decision aiding automation)? 
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2. METHOD 


2.1. Participants 

Twenty- three participants took part in the experiment. They consisted of: 

• Four (4) Commercial pilots at GTCC: This group had total flying time between 
456 - 1 185 hours, with a mean of 719.85 hours. The expert group had three flight 
instructors and a former military fighter pilot with commercial license. The age 
ranged from 27 to 43 with an average age of the expert pilots (EXPERT) 37.55 
years. 

• Seven (7) student pilots at GTCC . The students have some flight training and 
were familiar with computer-based flight simulators. The student age ranged from 
19 to 24 with an average age of the student pilots (SP) was 23.13 years. 

• Twelve ( 1 2) non-pilots (NP). This group were graduate and undergraduate 
students from North Carolina A&T State University. The NP group had no flight 
experience, except, in some cases, playing games with Microsoft Flight 
Simulation Software. The age range was 17.5 to 26 with an average age of 21 .64 
years. 

The participants were paid as follows: $20.5/hour for commercial pilots, $9.0/hour for 

student pilots; and $7. 5/hour for non-pilots selected from the student body at North 

Carolina A&T State University and Guilford Technical Community College (GTCC). 

The participants were paid full compensation only after completing the experiment as 

stipulated in the agreement. 

2.1. Flight Simulator 

The Professional version of Microsoft Flight Simulator 2000 was used to 
reconfigure Canadair 415 flight simulator located at GTCC. The Professional Edition 
software includes various aircraft with instrument panels, virtual cockpits, and exterior 
3D models. Figure 1 shows a photographic rendering of the Canadair 415 cockpit system. 
The two-pilot flight deck is fitted with a Honeywell EDZ-605 EFIS electronic flight and 
information system and dual air data computers. The instrument panels have a three-tube 
Integrated Instrument Display System and an electronic attitude director indicator (EADI) 
and electronic horizontal situation indicator (EHSI). The cockpit is fitted with a 
Litef/Honeywell attitude heading reference system (AHRS) and a Honeywell radio 
altimeter. The communications systems include a Global multiband radio 
communications set covering VHF/UHF/AM/FM bands, Rockwell Collins HF radios 
with two transponders and an emergency locator transponder (ELT). 


Figure 1 . A photographic rendering of Canadair 415 cockpit system. 

The weather system provided by the software dramatically improves the variety of 
weather as a user flies and the effects they see like clouds, precipitation, lightning, and 
more. This is very helpful in adding various complexities to the flight. Sound files were 
recorded and used in conjunction with the three scenarios to provide Air Traffic Control 
(ATC) commands and instructions. The Black Box application runs in conjunction with 
Flight Simulator 2000. It enables the user to record variables simultaneously, such as, 
airspeed, altitude, and heading in 10 second intervals. Figure 2 gives an illustration of a 
reconfigured cockpit with sample weather in the horizon. The basic hardware 
requirement consists of: (a) Two computer monitors: One monitor displays the terrain 
and weather while the second monitor displays the aircraft functionalities. This setup 
permits the experimenter to have a full control of the environment such as changing 
tasks, flight parameters and so on, (b) Input device: this consist of a Gameport joystick 
and keyboard; and (c) sound output device for communication with ATC. 
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Figure 2. Candair 41 5 airplane cockpit display rendering with sample weather in the 
horizon. 


2.3 Procedure 

The experiment was divided into six sections: 

(1) Introduction: In the introduction phase, the experimenter explained the purpose of 
the study and the risk involved. Prior to full participation, the participants were 
asked to read and sign a consent form. The participation of student pilots was 
required for GTCC class, other participants were voluntary. 

(2) Training: In this phase, the subjects were introduced to the concept of Free Flight, 
aircraft separation procedures, cockpit layout, instruments, weather conditions, 
and displays. Only the relevance cockpit instruments needed for the flying tasks 
were elaborated. The subjects also learn to use the joystick for navigation. The 
non-pilot subjects were introduced to Instrument Flight Rules (IFR) and Visual 
Flight Rules (VFR), and conditions that mitigate instrument meteorological 
conditions (IMC). All participants were also briefed on flight plans before actual 
flight. 

(3) Preliminary Learning of Flight Task Scenarios: In this phase, the subjects were 
allowed to fly in any of the scenarios while practicing aircraft separation tasks. 
The practice lasted 40 minutes for non-pilots, 25 minutes for student pilots, and 
1 0 minutes for commercial (expert) pilots. 
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(4) Audio Comprehension Test: This test was a part of training to ascertain how the 
subjects understood ATC call sign commands and procedures called in by voice. 
All participants used in the experiment passed the audio test. 

(5) Actual Flight Experiments: Each subject was supervised to perform a total of 96 
tests (4 trials under two traffic densities, two proximity of intruder aircraft, two 
tasks (climb or descent), and three weather scenarios). Each experimental trial 
was set at 10 minutes each, resulting in 16 hours per participant. In each flight 
scenario, the participants performed either a descending (approach to land) or 
climbing task. Conflict aircraft were fixed at an acute angle of 45 degrees based 
on the previous studies which indicate higher conflict detection and separation 
performance (Cashion & Lozito, 1999). During the experiments, the participants 
(owner aircraft) had the maneuvering responsibility according VFR right-of-way 
(ROW) rule. This rule stipulates that the conflict aircraft is on the right according 
to Federal Aviation Regulation (FAR) 91.113. Each participant used 2.5 hours per 
day with 5 minutes rest interval between trials. 

(6) At the end of each test, the subjects were asked to provide after-fact debrief by 
filling out scenario experience questionnaire (SEQ) (Appendix A). 

2.4. Weather Scenario Configuration 

The sample weather configuration with flight task representing scenarios is shown 
in Table 1 . Three primary scenarios based on cloud ceiling, precipitation, wind 
conditions, and visibility were tested. These scenarios are shown in Table 2. 


Table 1 : Sample Weather Conditions 


Cloud 

ceiling 

Precipitation 

Wind 

Visibility 

Traffic 

density 

Intruder 

aircraft 

proximity 

Flying 

task 

800 ft 

Light rain 

Wind 
gust = 
low 

8 miles 

(low, 

high) 

(near, far) 

(descend, 

climb) 

1800ft 

Moderate 

rain 

Wind 
gust = 
moderate 

4miles 

(low, 

high) 

(near, far) 

descend, 

climb) 

4000ft 

Rain with 
snow 

Wind 
gust = 
high 

1 mile 

(low, 

high) 

(near, far) 

descend, 

climb) 


2.5. Task 

Each participant was asked to fly under free flight with the preset weather 
scenarios. They were told that they are responsible for monitoring the status of the 
aircraft display for situation awareness and that they are free to contact ATC anytime 
during the flight. The flight instructors at GTCC served as ATC personnel. The 
participants were told that within the same altitude, there would be some aircrafts 
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(intruders) that may or may not fly closer to their (ownership) aircraft. If this should 
occur, they were to try to separate aircraft from that of the intruder. 


Table 2: Experimental Test Scenarios 


Test Scenario 

Information Elements 

Scenario- 1 

Scenario- 1 Data 
Cloud ceiling = 800 ft 
Precipitation = light rain 
Wind gust = low 
Visibility = 8 miles 

Scenario-2 

Scenario-2 Data 
Cloud ceiling = 1800 ft 
Precipitation = moderate rain 
Wind gust = moderate 
Visibility = 4 miles 

Scenario- 3 

Cloud ceiling = 4000 ft 
Precipitation = rain + snow 
Wind gust = high 
Visibility = 1 mile 


They were also told that within a weather scenario, the traffic density could vary between 
2-6 intruder aircraft (low TD) or between 7-12 intruder aircraft (high TD); the intruder 
proximity (Proximity) to the ownership aircraft can vary between 1-3 miles (near 
proximity) or between 4-8 miles (far proximity). As a part of flight plan, the participants 
were to be informed of weather conditions by ATC as follows: pre-waming of weather 
locations: (a) close to weather location (1-4 miles), moderate closeness to weather 
location (4-6 miles), and far from weather location (6-10 miles), and random weather 
occurrence without warning. 

After getting clearance from ATC, subjects took-off and climbed to the cruise altitude of 
4000 feet. The subjects were allowed to cruise for approximately 3-5 minutes. At the end 
of the cruise, the aircraft was programmed to generate a weather scenario, traffic density 
in the airspace, and the conditions requiring the participants to conduct aircraft separation 
tasks. The experimenter set and controlled 52 weather pre-waming and 24 random 
weather occurrences during the experimental cycle of a participant. Before each new trial, 
the air traffic was cleared and the flight plan re-initialized. The ATC used radio 
announcement with cockpit enunciator to broadcast separation risk and weather when 
necessary. 
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3. EXPERIMENTAL DESIGN AND RESULTS 

3.1. Dependent and Independent Measures 

The independent measure consisted of the weather scenarios as defined in 
Table 2. Each weather scenario was crossed with two levels of traffic density (the number 
of other aircrafts in the altitude as the ownership aircraft), two levels of intruder aircraft 
proximity, and two flying tasks. The effect of flight experience was also an independent 
variable. The dependent variables of interest were percentage detection accuracy of 
intruding aircraft, conflict detection time, and the frequency the participants flew into 
adverse weather with and without pre-warning. All participants took part in all 
experimental conditions resulting in within-subject experiment. 


3.2. Task 1: Aircraft separation procedure: What will pilots and controllers do when 
they need to separate aircraft from traffic and weather events. 


3.2.1 Approach 

In order to answer the above question, a set of questionnaires was designed to 
gather information on the protocols used by the participants when faced with aircraft 
separation tasks. In this study, an attempt was made to delineate behaviors that can be 
quantified and those that cannot. Skinner (1953) observed that behavior is difficult to 
compute, not because it is inaccessible, but because it is extremely complex. It is a 
process, rather than a thing, cannot be held still for observation (p.15)”. The major 
premise for using a behavior model is that people exhibit certain behavior tendencies in 
expectation of some reward (Blanford, 1993). An example is risk avoidance behavior 
associated with imminent collision of two aircraft, a typical incident due to self- 
separation task (Edwards, 1977). It is a great advantage to suppose that the probability 
that a response will occur ranges continuously between all or some causes. If we know 
what causes pilots to behave in a certain way, by discovering and analyzing these causes, 
we can predict behavior and to the extent that we can manipulate such behaviors to 
improve design. 

Pilots can exhibit one or several behavioral tendencies. For example, behavior can 
be intentional or goal-directed, reactive as in responding to information cues or stimuli, 
reflexive, as in an instant or automatic response to situations (Schneider & Schriffin, 
1977); and enactive, as governed by procedures and rules such as given in FAR 91.1 13 or 
FAA-S-8081-14 (1995), available at http://afts600.faa.gov/data/pract icalteststandard/faar 
s-8081-14.pdf . A pilot can practice all or some of the behaviors during an aircraft self- 
separation task. One of the several ways to represent information about these behavior 
schemes is by schema. A schema is compiled of selected pieces of behaviors; typically in 
a ranked priority order according to some attributes, such as, the saliency of information, 
level of perceived risk, and the urgency or criticality of incidents (Scholl, 1987). 

The following schema were identified to represent the pilot and controller task 
behaviors during aircraft self-separation tasks: 
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Recognition Schema: This schema represents the ability of agents (pilots and ATC operators) 
to recognize conflict incidents. Such incidents as identified by priority by the participants in the 
study are: 

• Converging headings. 

• Aircraft in the same altitude. 

• Speed differences between the owner and intruder aircraft. 

• Ability to predict or guess the relative position of the intruder aircraft relative to 
the ownership aircraft. 

Spatio-temporal Schema: This schema represents the judgment of space and time with 
respect to ownership and intruder aircrafts. The cardinality consist of 

• Positions 

• Altitudes. 

• Trajectories. 

• Time 

Aircraft position can be determined by the minimum lateral separation (MLAS), 
minimum height difference (MHD), and minimum longitudinal separation (MLOS). The 
minimum lateral separation is when the intruder aircraft is at the opposite direction traffic 
within the same route; MHD occurs when aircraft are at different flight levels in different 
directions in the same airspace; and, MLAS is the distance between consecutive aircraft 
at the same flight level on the same route, but the faster aircraft can overtake slower one 
subject to adequate MLAS and MHD. 

Conflict Reconciliation Schema: This schema represents the pilot and ATC conflict 
management behaviors, and is a function of many task attributes, including but not 
limited to, 

• Space available for vectoring. 

• Available altitudes. 

• Aircraft type and capabilities. 

• Proximity of the aircraft. 

• Distance of the aircraft to its destination. 

Decision Schema: This represents the decision-making behavior of the pilot and ATC 
under risk and time constraint. Some components of this behavioral schema are: 

• Determining intruder’s pilot intent. 

• Negotiating for airspace right of way. 

• Closing angles. 

• Planning and deciding on diversion of aircraft. 

• Dead reckoning. 

• Seeking advise from ATC or aircraft automation aid. 

3.2.2. Data Collection and Results 

A self-report questionnaire was administered to each of the participants at the end 
of the flying tasks. The first part of the questionnaire (Appendix A) was used to 
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determine the participant’ perception of cognitive tasks related to aircraft self-separation. 
The participants were asked to rate the knowledge content of the tasks according to its: 
Important (3), Relevance (2), and Concern (1). The question asked was: During the 
aircraft self-separation, rate the following cognitive tasks by its im portance, 
relevance, or concern as it pertains to task performance: 

(a) . Recognition of conflict incidents. 

(b) . Knowledge of intruding aircraft. 

(c) . Conflict reconciliation between intruder aircraft and ownership aircraft. 

(d) . Intruder pilot intent. 

The total weight for each cognitive variable was obtained by multiplying the number of 
respondents by the weights of the semantic attribute scores. Table 3 gives the result by 
the category of participants. Figure 3 shows the percent plot of the results. 


Table 3: Subjective Rated Scores of Cognitive Tasks As Perceived By Participants 
During Aircraft Separation Tasks 


Participant 

Recognize 
W 

Knowledge 

Reconcile 

Intent 

Expert (4) 

4*2=8 1 } 

4*3=12 

4*1=4 

4*1=4 

Student Pilot (7) 

4*3=12 

4*2=8 

3*1=3 

1*1=1 

Non-Student pilot (12) 

8*3=24 

10*2=20 

6*1=6 

5*1=5 

Total Score 

44 

40 

13 

9 


(*): # of Respondents * Score 


As shown in Table 3, having the spatial knowledge of the intruding aircraft was 
more important to the expert pilot, while the ability to recognize the intruding aircraft 
was rated relevance. Both the intent of the intruding pilot and conflict reconciliation were 
viewed as only a concern to the expert pilot, but not worrisome to impact task 
performance. The novices (student and non-student) pilots on the other hand indicated 
that the ability to recognize conflict incidents was more important, while the spatial 
knowledge of the intruding aircraft was deemed relevance. As with the expert pilots, 
conflict reconciliation and the intent of the intruder pilot were deemed as only a concern. 
Figure 3 shows the merit scores for all participants. In general, the ability to recognize the 
conflict incidents, followed by the ability to acquire the spatial location of the intruder 
aircraft relative to ownership aircraft were judged to be the major cognitive tasks as 
perceived by the participants during aircraft self-separation. 
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Figure 3: Percentage plot of participants’ perception of cognitive tasks in aircraft self- 
separation. 

3.2.2.I. Rating Incident Recognition And The Utility of CDTI 


Table 4 shows the percentage ratings of the participants’ ability to recognize 
conflict incidents using the available cognitive resources. 


Table 4: Percentage Utilization of Resources By Participants During Conflict Incident 


Participants 

ATC 

CDTI 

SELF | 

Expert 

8.5 

33.6 

57.9 

Student Pilot 

51.3 

34.3 

14.4 

Non-Student Pilot 

66.5 

26.4 

7.1 


Figure 4 illustrates these subjective perception scores. As shown in Figure 4, the 
experts tend to rely on their mental models in recognizing conflict incidents and are less 
likely to consult the ATC. However, they tended to use the CDTI as an assistant for 
situation awareness. The novice pilots were more likely to consult the ATC operators for 
conflict recognition tasks, used CDTI for information search and discovery, and were less 
likely to recognize aircraft conflicts without some support. 
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Figure 4. A graphical display showing the mean rating scores of participants’ use of 
cognitive resources for conflict recognition tasks. 

A meta-analysis was conducted on the recognition schema to determine 
which tasks were most important to the participants. Figure 5 shows the response by the 



Figure 5. Mean saliency scores of recognition tasks as perceived by the 
participants. 
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As shown in Figure5, the expert pilots were more concerned with the possibility 
of collision (convergence) of ownership and intruder aircraft (all 4 pilots score the 
maximum of 3 points). The position and speed of the aircraft were not concerns to the 
expert pilots. The student pilots were concerned with the position of their aircraft, 
followed by the possibility of their convergences. Both speed and altitude were scored 
between concerned and importance. The non-pilot participants were worried about the 
speed of the intruder aircraft and less worried about the altitude. The intruder aircraft 
position and converging headings were also concerns. 

3.2.2.2. Rating of Spatio-Temporal Tasks With Respect To Resources Used 

Table 5 shows the percentage ratings of the participants’ ability to judge events in 
space and time using the available cognitive resources. 


Table 5: Percentage Utilization of Resources By Participants During Spatio-Temporal 
Judgment Tasks 


Participants 

ATC 

CDTI 

SELF 

Expert 

5.3 

45.6 

50.9 

Student Pilot 

39.3 

40.2 

20.5 ^ 

Non-Student Pilot 

80.5 

14.7 

4.8 



ATC COT! SB.F 


Resources 


Figure 6. A graphical display of mean rating scores of the participants’ use of cognitive 
resources during spatio-temporal judgment tasks. 

Figure 6 illustrates these subjective perception scores. As shown in Figure 6, the 
expert pilots tended to rely on their mental models during spatial task executions and 
were less likely to consult the ATC. However, they tended to use the CDTI as an assistant 
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for situation awareness support. The student pilots tended to split their dependency on 
both the ATC and CDTI. The non-pilots showed high dependency on the ATC for spatial 
and navigation task executions (about 80.5 % of the time). 


A meta-analysis was conducted on the spatial memory tasks to determine which 
tasks were more important to the participants. Figure 7 shows the response by the 
participants. As shown in Figure 7, the expert pilots were more concerned with altitude 
and trajectory maneuvering than the position and time of the aircrafts. The student pilot 
showed high concern on the relative position of the intruder aircraft, and some concern 
on the spatial tasks. The non-pilot participants were worried about the position, speed, 
and trajectories of the intruder aircraft and some concern about the altitude. 


£ 

o 

o 


o 

c 

• 

T5 

C/3 



Expert S- Pilot N-Pilot 


B Position 
B Altitudes 

□ Trajectories 

□ Time 


Experience level 


Figure 7. Mean saliency scores of spatio-temporal judgment task as perceived by the 
participants 


3.2.2.3. Rating of Conflict Reconciliation Schema 

Table 6 shows the percentage ratings of the participants’ ability to resolve 
conflicts using the available cognitive resources. 


Table 6: Percentage Utilization of Resources by Participants During Conflict 
Reconciliation Tasks 


Participants 

ATC 

CDTI 

SELF 

C 

Expert 

20.1 

10.3 

69.6 

Student Pilot 

64.7 

3.8 

31.5 

Non-Student Pilot 

96.4 

0 

3.6 


Figure 8 illustrates these subjective perception scores. As shown in Figure 8, the 
expert pilots tended to rely on their personal knowledge to conflict management with the 
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intruder pilots (69.6%); and 20.1% of the time, the expert pilot were likely to contact 
ATC for help. The student pilot depended on ATC about 65% of the time, rarely used the 
CDTI, and 32% of the time, they depended on personal conflict management skills. The 
non-pilots depended almost entirely on ATC. The results showed that the participants 
rarely used CDTI during conflict management related to aircraft separation. 



ATC CDTI SELF 


Figure 8. A graphical display of mean rating scores on participants use of cognitive 
resources for conflict resolution tasks. 

A meta-analysis was conducted on the conflict management schema tasks to 
determine which tasks were more important to the participants. Figure 9 shows the 
response by the participants. 
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Figure 9 . Mean saliency scores of conflict management tasks as perceived by the 
participants. 


As shown in Figure 9, the expert pilots were more concerned with altitude and vectoring; 
aircraft type was a high concern, and aircraft proximity was a concern. The distance 
between intruder and ownership aircraft was somehow worrisome to the expert pilot. The 
student pilots indicated absolute important to altitude availability and proximity of the 
aircrafts, high concerns for distance and vectoring, and little worrisome on aircraft type 
and capabilities. The non-student pilots showed absolute important to vectoring, 
proximity, and distance between the aircraft. Concern was also indicated for aircraft type 
and capabilities. 


3.2.2.4. Rating of Decision Schema 

Table 7 shows the percentage ratings of the participants’ decision-making during 
risk as induced by adverse weather and/or aircraft convergence. 


Table 7: Percentage Utilization of Resources by Participants During Decision-making 
Tasks. 


Participants 

ATC 

CDTI 

SELF 

Expert 

5.3 

40.7 1 

54.0 

Student Pilot 

28.5 

53.1 

18.4 

Non-Student Pilot 

45.7 

48.2 

6.1 


Figure 10 illustrates these subjective perception scores. As shown in Figure 10, 
the experts tended to rely on their mental models (54%) and on CDTI (40.7%) in 
decision-making behaviors. The expert pilots interaction with ATC was very low (5.3%). 
The student pilot depended on CDTI about half the time (53.1%) and about one quarter of 
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the time on ATC (28.5%), and depended on personal judgments about 18.4% of the time. 
The non-pilots depended almost equally on CDTI (48.2%) and ATC (45.7%) with 6.1% 
on personal judgment. The results showed that the participants use CDTI highly during 


decision-making tasks. 



Figure 10. A graphical display of mean rating scores of participants use of cognitive 
resources during decision-making tasks. 

A meta-analysis was conducted on the conflict management schema tasks to 
determine which was more important to the participants. Figure 1 1 shows the response by 
the participants. 
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Figure 1 1 . Mean saliency scores of decision-making tasks as perceived by the 
participants. 

As shown in Figure 1 1 , the expert pilots were more concerned with interpreting 
the intruder pilot intent; as well as negotiating for airspace right of way (ROW); they also 
showed concerns on closing angles and above worrisome scores for dead reckoning and 
diversion of aircraft. The student pilots indicated high concern on closing separation 
angles and aircraft diversion, and concern on intruder pilot intent, right of way, and dead 
reckoning. The non-student pilots showed high concern on closing angles and aircraft 
diversion; slight concern on right of way and dead reckoning tasks, and some worry on 
dealing with intruder pilot intent. 

3.3. Task 2: Functionality of aircraft separation: Will free flight separation be 

practical with weather events in the airspace? 

3.3.1. Analysis of Conflict Detection Accuracy Data 

The flight data for conflict detection performance on each weather scenario was 
summarized as shown by means and standard deviations on Tables 8-10. From now on, 
non-pilot data will not be used for further analysis. The intent is to only compare 
participants with some flight experience with commercial pilots. The data was used as a 
control to compare behavioral variables in Taskl only. 
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Table 8: Mean and Standard Deviation Summary of Percentage Accuracy for Scenario- 1 


Experience 

Traffic 

Low 

Density 

High 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

97.84 

(1.26) 

98.29 

(1.46) 

98.75 

(1.33) 

99.2 

(2.0) 

97.31 

(1.64) 

97.20 

(1.29) 

95.40 

(1.99) 

94.8 
(1 .61) 

Student Pilots 

78.42 

(3.85) 

80.92 

(4.21) 

81.26 

(2.34) 

84.37 

(2.92) 

65.21 

(6.0) 

63.42 

(3.64) 

67.3 
(4- 12) — 

66.91 

(2-36) 

Non Pilots 

60.3 

(7.51) 

68.2 

(6.24) 

63.8 

(7.93) 

67.79 

(8-41) 

59.30 

(9-45) 

56.35 

(6.90) 

63.41 

(10.15) 

60.18 

(7.16) 


(mean, std.) 


Table 9: Mean and Standard Deviation Summary of Percentage Accuracy for Scenario-2 


Experience 

Traffic 

Low 

Density 

High 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

97.54 

(2.61) 

95.16 

(1.52) 

98.61 

(1.84) 

97.3 

(2.66) 

95.66 

(1.97) 

95.24 

(2.64) 

94.64 

(1-99) 

96.21 

(2.64) 

Student 

Pilots 

71.38 

(4.02) 

75.2 

(5.68) 

73.69 

(5.13) 

72.11 

(6.02) 

63.46 

(4.16) 

61.24 

(4-92) 

68.11 

(5.130) 

63.89 

(3-75) 

Non Pilots 

62.22 

(5.21) 

63.19 

(4.67) 

60.22 

(5.68) 

56.29 

(7-14) 

55.24 

(3.20) 

52.10 

(4.29) 

60.33 

(5.74) 

56.24 

(9.34) 


(mean, std.) 


Table 10: Mean and Standard Deviation Summary of Percentage Accuracy for Scenario-3 


Experience 

Traffic 

Low 

Density 

High 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

92.70 

(1.66) 

94.96 

(2.41) 

93.25 

(1-09) 

96.33 

(1.24) 

94.39 

(2.61) 

92.64 

(1.68) 

96.48 
(2-63) _ 

95.53 

(1.97) 

Student 

Pilots 

68.71 

(3.84) 

72.33 

(5.11) 

71.40 

(6.00) 

70.89 

(5.08) 

60.32 

(3.89) 

56.24 

(7.24) 

56.82 

(6.14) 

59.37 

(6.41) 

Non Pilots 

43.18 

(10.42) 

40.66 

(1251) 

41.44 

(7.41) 

45.91 

(10.88) 

36.24 

(5.62) 

38.92 

(H.24) 

40.15 

(11.96) 

Bi 


(mean, std.) 


3.3.1. 1. Effect of Weather Conditions on Percentage Detection Accuracy 

33.1.1.1. Scenario-Based Performance Differences 

The null hypotheses investigated for each weather scenario are as follows: 
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Hoi: Both the commercial (Expert) and student pilots will perform equally in mean 
percentage detection accuracies. 

H 0 2 : The levels of traffic density will not affect percentage detection accuracies. 

H 03 The proximity levels of the ownership and intruder aircraft will not affect percentage 
detection accuracies. 

H 0 4 : There are no mean differences between percentage detection accuracies during 
climbing or descending tasks 

A four-way within-subject, 2X2X2X2 ANOVA technique was used to analyze the data. 
The data were analyzed with the SAS software package (Dilorio, 1991). 

Results for Scenario-1 

The result of ANOVA is shown in Table 1 1 . The experience levels of the pilots were 
(F (1,48)=5.04, p <0.001); Traffic density was significant (F (1,48) = 1 1.59, p <0.028); 
Proximity of aircraft was significant (F (1,48) =13.1; p<0. 001). There were also some 
interaction effects: Experience level of pilots and traffic density 
(F (1,48) = 5.89; p<0. 049); Experience level of the pilots and proximity of the aircrafts 
(F (1,48) = 4.958; p<0. 0036); traffic density and proximity of the aircrafts (F (1,48) = 
9.48; p<0. 001); Experience levels of the pilots, traffic density, and proximity (F (1,48) = 
5.37; p<0. 0060; Traffic density, proximity, and the flying tasks (F (1,48) = 4.73; p<0. 
0037). 


Table 1 1: ANOVA Results for Scenario-1 Based on Analysis of Percentage Detection 


Accuracy Data (F (1,48; a - 

= 0.05) = 

=4.08) 




Source 

DF 

SS 

MS 

F value 

Pr>F 

Experience (E) 

1 

9.04 

9.04 

5.04 

0.001 

TD 

1 

21.34 

21.34 

11.9 

0.028 

Proximity 

1 

23.51 

23.51 

13.1 

0.001 

Task 

1 

2.938 

2.938 

1.85 

0.049 

E*TD 

1 

10.57 

10.57 

5.89 

0.001 

E*Proximity 

1 

4.958 

4.958 

4.958 

0.0036 

E*Task 

1 

3.163 

3.163 

1.764 

0.193 

TD*Proximity 

1 

16.979 

16.979 

9.48 

0.001 

TD*Task 

1 

6.473 

6.473 

3.59 

0.38 

Proximity* Task 

1 

3.781 

3.781 

2.109 

0.08 

E*TD*Proximity 

1 

9.63 

9.63 

5.37 

0.006 

E*TD*Task 

1 

1.508 

1.508 

0.841 

0.169 

TD*Proximity*Task 

1 

8.48 

8.48 

4.73 

0.037 

E*TD*Proximity *Task 

1 

3.478 

3.478 

1.94 

0.26 

Error 

48 

86.064 

1.793 



Total 

62 

211.912 




i * 

Results for Scenario-2 

The result of ANOVA for scenario-2 is shown in Table 12. The experience levels of the 
pilots were significant (F (1,48)=4.51, p <0.0001); Traffic density was significant 
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(F (1,48) = 4. 19; p <0.035). There were also some interaction effects: Level of experience 
of pilots and traffic density (F (1,48) = 5.33; p<0.001); Experience level of the pilots and 
proximity of the aircrafts (F (1,48) = 4.81 ; p<0.021); traffic density and proximity of the 
aircrafts (F (1,48) = 4.26; p<0.0001); Experience levels of the pilots, traffic density, and 
proximity (F (1,48) = 4.68; p<0. 001 ; Traffic density, proximity, and the flying tasks 
(F (1,48) = 5.18; p<0. 0219). 

Results for Scenario-3 

The result of ANOVA for scenario-2 is shown in Table 13. The experience levels of the 
pilots were significant (F (1,48)=4.18, p <0.001); Traffic density was significant 
(F (1,48) = 4.52; p <0.008). There were also some interaction effects: Level of experience 
of pilots and traffic density (F (1,48) = 4.26; p<0.007); Experience level of the pilots and 
proximity of the aircrafts (F (1,48) = 5.69; p<0.001); traffic density and proximity of the 
aircrafts (F (1,48) = 4.19; p<0.06); Experience levels of the pilots, traffic density, and 
proximity (F (1,48) = 5.1 1; p<0. 001; Traffic density, proximity, and the flying tasks 
(F (1,48) = 4.27; p<0. 016); and interaction between all four levels of the main effect 
(F = 4.18; p<0. 041). 

3.3.1. 1.2. Analysis of Effects of Weather Scenarios 

The null hypothesis investigated is that there are no statistical significant 
differences in percentage detection accuracy between the weather scenarios. 

Table 12: ANOVA Results for Scenario-2 Based on Analysis of Percentage Detection 




24 


Table 13: ANOVA Results for Scenario-3 Based on Analysis of Percentage Detection 
Accuracy Data (F (1,48; a - 0.05) =4.08) 


Source 

DF 

ss 

MS 

F value 

Experience (E) 

1 

9.841 

9.841 

4.18 

TD 

1 

10.622 

10.622 

4.52 

Proximity 

1 

7.56 

7.56 

3.21 

Task 

1 

4.311 

4.311 

1.83 

E*TD 

1 

10.04 

10.04 

4.26 

E*Proximity 

1 

13.41 

13.41 

5.69 

E*Task 

1 

8.552 

8.552 

3.63 

TD*Proximity 

1 

9.872 

9.872 

4.19 

TD*Task 

1 

1.571 

1.571 

0.667 

Proximity* Task 

1 

6.22 

6.22 

2.64 

E*TD*Proximity 

1 

12.04 

12.04 

5.11 

E*TD*Task 

1 

4.453 

4.453 

1.89 

TD * Prox imity * T ask 

1 

10.06 

10.06 

4.27 

E*TD*Proximity* Task 

1 

9.848 

9.848 

4.18 

Error 

48 

113.09 

2.356 


Total 

62 

231.49 




The data was analyzed with ANOVA with task data (climbing and descending) 
blocked since earlier analysis indicated that weather did not show any significant effect 
on flying tasks. The result was significant leading to the rejection of the null hypothesis 
(F(2,144) = 23.81; p , 0.0001) > 19.5 (F(2,144; a = 0.05). For climbing task, there were 
interactions between Scenario- 1 and Scenarion-2 using the expert data as observed at two 
traffic densities ( see Figure 12). There was no obvious interaction for descending task 
under the same conditions (Figure 13). Scenario-3 showed degrading performance in all 
conditions. 

For the student pilots, there was significant interaction between Scenario- 1 and 
Scenario-2 for climbing task under high traffic density (Figure 14). No obvious 
interactions were observed descending task (Figure 15). Again, Scenarion-3 showed the 
worst performance. 


Pr>F 

0.001 

0.008 

0.033 

0.26 

0.007 

0.001 

0.029 

0.006 

0.083 

0.37 

0.001 

0.216 

0.016 

0.041 




% detection accuracy d % detection accuracy 
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•e 12. Mean percentage detection accuracy by expert pilots on climbing task. 
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Figure 13. Mean percentage detection accuracy by expert pilots on descending task 
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Low-TD High-TD 

Traffic density (TD) 


Figure 14. Mean percentage detection accuracy by student pilots on climbing task. 
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Figure 15. Mean percentage detection accuracy by student pilots on climbing task. 

3.3.2. Analysis of Conflict Detection Time Data 

The flight data for conflict detection time in each scenario was summarized as 
shown by means and standard deviations on Tables 14-16. 
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Table 14. Mean and Standard Deviation Summary of Detection time for Scenario-1 


Experience 

Traffic 

Low 

Density 

High 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

110.6 

(6.24) 

96.24 

(8.29) 

98.64 

(6.67) 

119.2 

(6.92) 

116.29 

(5.62) 

121.41 

(11.49) 

106.34 

(15.14) 

101.43 

(9-21) 

Student 

Pilots 

136.54 

(10.94) 

128.33 1 
(15.66) 

133.21 

(19.24) 

120.4 

(10.65) 

138.62 

(18.24) 

136.24 

(21.24) 

120.66 

(20.09) 

130.92 

(18.42) 

Non Pilots 

138.6 

(11.62) 

124.93 

(20.44) 

136.4 

(21.39) 

110.41 

(6.9) 

140.36 

(12.35) 

13866 

(14.25) 

136.21 

(28.6) 

133.7 

(21.31) 


(mean, std 


Tablel5: Mean and Standard Deviation Summary of Percentage Accuracy for Scenario-2 



Traffic 

Low 

Density 

HiSt 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Experience 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

121.66 

(4.6) 

104.62 

98.24) 

108.1 

(6.33) 

120.36 

(5.89) 

136.45 

(6.24) 

138.11 

(9.33) 

123.6 

(6-42) 

128.43 

(12.62) 

Student 

Pilots 

139.21 

(11.45) 

140.91 

(8.01) 

137.45 

(7.45) 

135.63 

(12.31) 

140.33 

(6.240 

139.01 

(8.39) 

142.33 

(10.14) 

136.84 

(5.36) 

Non Pilots 

143.24 

(7.38) 

145.6 

(6.41) 

139.33 

(8.24) 

139.8 

(7.31) 

146.24 

(8.09) 

140.80 

(7.02) 

148.22 

(6.19) 

138.3 

(4.22) 


(mean, std.) 



Traffic 

Low 

Density 

_ High 

Near Proximity 

Far Proximity 

Near Proximity 

Far Proximity 

Experience 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Climb 

Descend 

Pilots 

128.51 

(10.42) 

126.67 

(15.30 

128.6 

(9.2) 

136.4 

(13.6) 

140.32 

(10.25) 

143.1 

(9.66) 

138.6 

(10.62) 

140.25 

(13.1) 

Student 

Pilots 

148.25 

(9.26) 

143.64 

(11.27) 

140.66 

(13.1) 

139.21 

(8.04) 

143.89 

(12.13) 

146.24 

(11.42) 

148.36 

(8.33) 

141.9 

(7-62) 

Non Pilots 

154.41 

(20.36) 

150.69 

(24.9) 

149.71 

(11.36) 

145.66 

(18.4) 

156.31 

(21.45) 

158.14 

(16.24) 

151.36 

(21.64) 

148.62 

(9.55) 


(mean, std.) 


3 .3.2.1. Effect of Weather Conditions on Conflict Detection Time 
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3 .3.2. 1.1. Scenario-Based Performance Differences 

The null hypotheses investigated for each weather Scenario are as follows: 

H 0 i: The mean conflict detection times are the same for both the commercial (Expert) and 
student pilots 

H 0 2 There are no statistical differences in mean conflict detection times for tasks 
performed in low or high traffic density conditions. 

H 03 The proximity levels of the ownership and intruder aircraft have no effect on mean 
conflict detection times. 

H 0 4 : There are no mean differences in mean conflict detection times during climbing or 
descending tasks 


Results for Scenario-1 

The result of ANOVA for Scenario- 1 is shown in Table 17. The experience levels 
of the pilots were significant (F (1,48)=6.27, p <0.0001); Traffic density was significant 
(F (1,48) = 4.18; p <0.0021). There were some interaction effects: Level of experience of 
pilots and traffic density (F (1,48) = 6.58; p<0.0019); Level of experience of pilots and 
tasks (F (1,48) = 4.29; p<0.006);traffic density and proximity of the aircrafts (F (1,48) = 
4.83; p<0.01); Experience levels of the pilots, traffic density, and proximity (F (1,48) = 
4.36 p<0. 0001 ; Traffic density, proximity, and the flying tasks (F (1,48) = 4.83; 

p < 0 . 001 ). 


Table 17: ANOVA for Scenario- 1 Conflict Detection Time 


Source 

Experience (E) 

TD 

Proximity 

Task 

E*TD 

E*Proximity 

E*Task 

TD*Proximity 

TD*Task 

Proximity*Task 

E*TD*Proximity 

E*TD*Task 

TD*Proximity*Task 

E*TD*P r0X imity*T aS k 

Error 

Total 


DF 

SS 

MS 

F value 

Pr>F 

1 

11.744 

11.744 

6.27 

0.0001 

1 

7.829 

7.829 

4.418 

0.0021 

1 

6.986 

6.986 

3.73 

0.167 

1 

3.58 

3.58 

1.91 

0.235 

1 

12.32 

12.32 

6.58 

0.0019 

1 

4.945 

4.945 

2.64 

0.096 

1 

8.04 

8.04 

4.29 

0.006 

1 

9.05 

9.05 

4.83 

0.001 

1 

3.901 

3.901 

2.104 

0.367 

1 

6.799 

6.799 

3.63 

0.001 

1 

8.167 

8.167 

4.36 

0.0001 

1 

5.638 

5.638 

3.01 

0.341 

1 

9.05 

9.05 

4.83 

0.001 

1 

6.874 

6.874 

3.67 

0.075 

48 

89.904 

1.873 



62 

194.827 





Results for Scenario-2 

The result of ANOVA for Scenario-2 is shown in Table 18. The experience levels 
of the pilots were significant (F (1,48)=5.66, p <0.001); Traffic density was significant (F 
(1,48) = 4.81; p <0.037). The proximity of intruder and ownership aircraft was significant 
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(F (1,48) = 4.37; p <0.043 There were also some interaction effects: Level of experience 
of pilots and traffic density (F (1,48) = 4.29; p<0.001); Experience level of the pilots and 
proximity of the aircrafts (F (1,48) = 5.38; p<0.051); traffic density and proximity of the 
aircrafts (F (1,48) = 4.36; p<0.01); type of task and proximity of intruder aircraft (F(l,48) 
= 6.13;p = 0.0001); Experience levels of the pilots, traffic density, and proximity (F 
(1,48) = 4.72 p<0. 01; Experience levels of the pilots, traffic density, and task (F (1,48) = 
5.23 p<0. 047);Traffic density, proximity, and the flying tasks (F (1,48) = 4.167; p<0. 
024); and interaction between all four levels of the main effect (F (1,48) = 4.38; p<0. 

006). 

Results for Scenario-3 

The result of ANOVA for Scenario-3 is shown in Table 19. The experience levels 
of the pilots were significant (F (1 ,48)=4.39, p <0.0413); Traffic density was significant 
(F (1 ,48) = 4.26; p <0.05 1 ). The proximity of intruder and ownership aircraft was 
significant (F (1,48) = 4.62; p <0.037. The proximity of intruder and ownership aircraft 
was significant (F (1,48) = 4.62; p <0.037. There were also some interaction effects: 

Level of experience of pilots and traffic density (F (1,48) = 4.834; p<0.039; Experience 
level of the pilots and proximity of the aircrafts (F (1,48) = 4.68; p<0.0001); 

Table 18: ANOVA for Scenario-2 Conflict Detection Time 


Source 

DF 

ss 

MS 

F value 

Experience (E) 

1 

12.1 

12.1 

5.66 

TD 

1 

10.284 

10.284 

4.81 

Proximity 

1 

9.343 

9.343 

4.37 

Task 

1 

6.094 

6.094 

2.85 

E*TD 

1 

9.172 

9.172 

4.29 

E*Proximity 

1 

11.502 

11.502 

5.38 

E*Task 

1 

3.763 

3.763 

1.76 

TD*Proximity 

1 

9.32 

9.32 

4.36 

TD*Task 

1 

4.04 

4.04 

1.89 

Proximity* Task 

1 

13.11 

13.11 

6.13 

E*TD*Pr 0X imity 

1 

10.09 

10.09 

4.72 

E*TD*Task 

1 

11.182 

11.182 

5.23 

TD*Proximity*Task 

1 

8.892 

8.892 

4.16 

E*TD*Proximity* Task 

1 

9.343 

9.343 

4.37 

Error 

48 

102.624 

2.138 


Total 

62 

230.861 




traffic density and proximity of the aircrafts (F (1,48) — 4.38; p<0.0471); type of task and 
proximity of intruder aircraft (F(l,48) = 4.29;p = 0.006); Experience levels of the pilots, 
traffic density, and proximity (F (1,48) = 4.94; p<0. 0001; Experience levels of the pilots, 
traffic density, and task (F (1,48) = 4.17; p<0. 047);Traffic density, proximity, and the 
flying tasks (F (1,48) = 4.167; p<0. 043); and interaction between all four levels of the 
main effect (F (1,48) = 4.07; p<0. 035). 


Pr>F 

0.001 

0.037 

0.043 

0.169 

0.001 

0.051 

0.33 

0.001 

0.395 

0.0001 

0.01 

0.047 

0.024 

0.006 
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Table 19: ANOVA for Scenario-3 Conflict Detection Time 


Source 

Experience (E) 

TD 

Proximity 

Task 

E*TD 

E*Proximity 

E*Task 

TD*Proximity 

TD*Task 

Proximity *Task 

E*TD*Proximity 

E*TD*Task 

TD*Proximity*Task 

E*TD*Proximity *Task 

Error 

Total 


DF 

ss 

MS 

F value 

Pr>F 

1 

13.771 

13.771 

4.39 

0.0413 

1 

13.364 

13.364 

4.26 

0.051 

1 

14.493 

14.493 

4.62 

0.037 

1 

6.067 

6.067 

1.934 

0.233 

1 

15.164 

15.164 

4.834 

0.039 

1 

14.681 

14.681 

4.68 

0.0001 

1 

3.645 

3.645 

1.162 

0.094 

1 

13.74 

13.74 

4.38 

0.0471 

1 

3.297 

3.297 

1.051 

0.429 

1 

13.458 

13.458 

4.29 

0.006 

1 

15.497 

15.497 

4.94 

0.0001 

1 

13.081 

13.081 

4.17 

0.043 

1 

11.33 

11.33 

3.613 

0.268 

1 

12.768 

12.768 

4.07 

0.035 

48 

150.576 

3.137 



62 

314.935 





3.3.2.I.2. Analysis of Effects of Scenarios On Conflict Detection Time 

he null hypothesis investigated is that there are no statistical significant 
differences in mean conflict detection times between the three weather scenarios. 

The data was analyzed with ANOVA using task data (climbing and descending) 
blocked since analysis indicated that weather did not show any significant effect on the 
type of flying task. The result was significant leading to the rejection of the null 
hypothesis (F (2,1 44) = 32.95; p , 0.0001) > 19.5 (F (2, 1 44; a = 0.05). 

The expert data for climbing task failed to show any interactions for all scenarios 
(Figure 16). Descending task showed apparent interaction at low-density traffic between 
Scenario-1 and Scenario-2 (Figure 17). Scenario-3 showed degrading performance 
(increase in detection task) for both traffic densities. 

For the student pilots, both climbing and descending tasks showed significant 
interactions between Scenario- 1 and Scenario-2 for high-density traffic conditions 
(Figures 18-19). Scenario-3 showed higher incident detection times. 
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e 16. Mean incident detection times by expert pilots on climbing task. 
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Figure 17: Mean incident detection times by expert pilots on descending task. 
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e 1 8: Mean incident detection times by student pilots on climbing task 
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Figure 19: Mean incident detection times by student pilots on descending task 
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3.4. Task 3: Separation Risk: Will pilots take more risks flying into weather under 

free flight decision making? 

This part of the study investigated risk behaviors of the free flight pilots using 
visual flight rules (VFR) and instrument meteorological conditions (IMC). The pilot’s 
decision to fly into adverse weather conditions was examined under two conditions: 
whether the pilot received a prior weather warning or no warning from the ATC. Under 
the prior warning, we examine the effect of distance to the location of the weather 
condition. Three distance levels were examined as follows: close to the weather event (1- 
4 miles), moderate closeness (4-6 miles), and far (6-10 miles). During the experiments, 
52 pre-warning weather conditions were issued by ATC and 24 no warning conditions 
were randomly generated by the flight simulator. The participants were informed to 
watch the CDTI for random weather conditions. The dependent variable was the number 
of times the participants continue to fly into weather with or without warning. 

3.4.1. Analysis of Expert (Commercial Pilot) Behaviors 

Figure 20 shows the plot of the percentage of times expert pilots flight into 
weather conditions for the three scenarios under prewaming and no warning conditions. 
Scenario-3 showed more avoidance behavior (i.e., less occurrence to fly into weather). 
All main scenarios were significant in mean proportion of times the expert pilots flew 
into adverse weather (yf = 8.39 ; 0.001 < (2, 0.025) = 7.378). There were interactions 

between all three scenarios. Scenario- 1 showed that pilots will fly more into adverse 
weather because of their perception of less risk. Scenario-2 leaves a puzzle as to why 
expert pilots will continue to fly into weather under no warning condition and less under 
prewaming condition. This observation needs further study for validation. 
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Figure 20: Display of mean percentage of cases expert pilots flew into adverse weather 
under different weather information conditions. 

3.4.2. Analysis of Student Pilot Behavior 

Figure 21 shows the plot of the percentage of times student pilots flew into 
weather conditions for the three scenarios under prewaming and no warning conditions. 
The main scenarios were significant in mean proportion the student pilots flew into 
adverse weather ( X 2 = 9.10; 0.017 < X 2 (2, 0.025) = 7.378). Scenario-3 showed more 
avoidance behavior (i.e., less occurrence to fly into weather). Scenario- 1 showed that the 
student pilots have the same probability of flying into an adverse weather in either 
prewaming or no warning situations. Again, Scenario-2, leaves a puzzle as to why 
student pilots will continue to fly into weather under no warning condition and less 
under prewaming condition. A further study to understand this unique results is desirable. 



Weath«r Information 

| _ ___ ___ -J 

Figure 21; Display of mean percentage of cases student pilots flew into adverse weather 

under different weather information conditions. 

3.4.3 Comparing Expert- and Student - Pilot Behaviors Under Pre-warning 
Weather Conditions 

Figure 22 shows the plot of the percentage of cases the expert- and student- pilots 
flew into weather conditions for the three scenarios under all prewaming conditions. The 
differences were significant for Scenario-2 and Scenario-3. The student pilots showed a 
decreasing slope in judgment to fly more into Scenario-1 weather and decreasing in that 
intent as the weather conditions became more adverse. The expert pilots showed more 
flight into Scenario- 1 weather and less into Scenario-2 weather. The expert’s decision to 
fly more into Scenario-2 than Scenario- 1 in no-waming condition remains elusive for 
explanation. One explanation, however, may be attributed to ability to estimate risk under 
this scenario O’Hare & Smitheram, 1995). 
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3.4.4. Comparing Expert- and Student-Pilot Behaviors Under No-warning Weather 
Conditions. 

Figure 23 shows the plot of the percentage of cases the expert- and student- pilots flew 
into weather conditions for the three scenarios under no warning conditions. The student 
pilots showed a decreasing slope in judgment to fly more into Scenario-1 weather and 
decreasing in that intent as the weather conditions became more adverse. The expert 
pilots showed less flight into Scenario- 1 and more into weather Scenario-2... The 
decision of the expert to fly more into Scenario-2 than Scenario- 1 posits the same 
dilemma in interpreting behavior as the observations under pre-warning condition and 
desires further studies. 



Figure 22. Comparing expert and student behaviors under prewaming weather conditions 
(52 Prewaming samples). 
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Figure 23. Comparing expert and student behaviors under no warning weather conditions 
(24 No warning samples). 

3.4.5 Effect of Weather Warning Distance On Decision to Fly Into Adverse 
Weather Conditions 

This section presents the result of effects of prewaming distance on the pilots’ 
decision to fly into weather conditions. The hypotheses were investigated with the data 
collected on the number of instances that the pilots flew into weather. The hypotheses 
are: 

Hoi: The prewaming distance to weather conditions have equal effects on the pilots’ 
decision to fly into weather conditions. 

Hq 2 : There are no statistical differences in the mean number of times the pilots will fly 
into weather conditions under the weather scenarios. 

Ho 3 :- There are no statistical differences in the mean number of times commercial pilots 
(experts) and student pilots will fly into weather conditions under the weather scenarios. 
Ho 4 ;. There are some interactions on pilots’ decision behaviors as mitigated by 
prewaming distance, weather scenario, and experience of the pilots. 

3.4.5.1 Results 

The data was analyzed using within subject 3X3X2 ANOVA.(Table 20). There 
were three levels of warning distance, three levels of weather scenarios, and two levels of 
flight pilots (commercial and students). The result was significant for all main effects: 
Distance (F( 2,54) = 3.94; p = 0.0001), Scenario (F(2,54) = 3.54); p = 0.0231), 

Experience (F(l,54), = 4.73; p = 0.39). The only interaction was between prewaming 
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distance and weather scenarios (F(4,54) = 3.97; p = 0.02). Figure 24 shows the mean 
number of times that expert pilots flew into weather by prewaming information distance. 
Duncan’s multiple-range test on the means show that the nearer the prewaming 
information to weather conditions, the less likely the pilots will fly into the condition. 
Duncan’s test revealed no statistical differences between moderate and far distance (with 
mean number of flight into weather of 7.33 and 7.67; p < 0.0025). Prewaming distance 
nearer to the location of weather showed an average of 2.33 times that the pilots flew into 
weather conditions. 


Table 20: AN OVA for Effect of Warning Distance Before Weather 
Using Number of Plan Continuation Events 


Source of 

Sum of 

df 

Mean 

F-value 

_e 

Variation 

Squares 


Square 



Distance (D) 

8.56 

2 

4.28 

3.93 

0.0001 

Scenario (S) 

7.72 

2 

3.86 

3.54 

0.0231 

Experience (E) 

5.17 

1 

5.17 

4.73 

0.039 

D*S 

17.32 

4 

4.33 

3.97 

0.02 

D*E 

5.34 

2 

2.67 

2.45 

0.063 

S*E 

2.26 

2 

1.13 

1.04 

0.01 

D*S*E 

5.96 

4 

1.49 

1.37 

0.038 

Error 

58.56 

54 

1.09 



Total 

110.89 

71 
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Figure 24. Mean distribution of the number of times expert pilots flew into weather by 
prewaming information distance. 
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Figure 25 shows the mean number of times that the student pilots flew into 
weather by prewaming information distance. Duncan’s multiple-range test on the means 
show that the nearer the pre-warning information to weather condition, the less likely 
they will fly into the condition. Duncan’s test revealed no statistical differences between 
moderate and far distance (with mean number of flight into weather of 8 and 7; p < 
0.0001). Prewaming distance nearer to the location of weather showed an average of 2.7 
times that the pilots flew into weather condition. 



Scenario- 1 Scenario-2 Scenario-3 

Weather scenario 


Figure 25. Mean distribution of the number of times student pilots flew into weather by 
prewaming information distance. 
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4. DISCUSSIONS AND SUMMARY 


4.1. Discussions 

4.1.1. Behavior Analysis of Pilots During Aircraft Separation Tasks 
The followings behaviors were observed: 

4.1.1.1. Expert Behavior: 

(i) The intent of the intruding pilot and conflict reconciliation were viewed as 
only a concern to the expert pilot, but not worrisome to impact task 
performance. 

(ii) The experts tended to rely on their mental models in recognizing conflict 
incidents and are less likely to consult the ATC. 

(iii) They tended to use CDTI for situation awareness. 

(iv) The expert pilots were more concerned with the possibility of collision 
(convergence) of ownership and intruder aircraft (all 4 pilots score the 
maximum of 3 points). 

(v) The position and speed of the aircraft were not concerns to the expert pilots. 

(vi) The expert pilots tended to rely on their mental models during spatial task 
executions and were less likely to consult the ATC. 

(vii) The expert pilots were more concerned with altitude and trajectory 
maneuvering than the position and time of the aircrafts. 

(viii) The expert pilots tended to rely on their personal approach to conflict 

management with the intruder pilots (69.6%); and 20.1% of the time, they 
more were likely contact ATC for help. 

(ix) The expert pilots were more concerned with altitude and vectoring; aircraft 
type was a high concern, and aircraft proximity was a concern. The distance 
between intruder and ownership aircraft was somehow worrisome to the 
expert pilot. 

(x) During critical decision making such as determining the intruder pilot’s intent 
or closing separation angle, the experts tended to rely on their mental models 
(54%) and on CDTI (40.7%) in decision-making behaviors. The expert pilots 
interaction with ATC was very low (5.3%). 

4.1. 1.2. Student Pilot Behavior: 

(i) The ability to recognize conflict incidents was more important, while the 
spatial knowledge of the intruding aircraft was deemed relevance. 

(ii) The student pilots were more likely to consult the ATC operators for help 
during conflict recognition tasks. 

(iii) They often used CDTI for information seeking, and were less likely to 
recognize aircraft conflicts without some support. 

(iv) The student pilots were concerned with the position of their aircraft, followed 
by the possibility of their convergence. 
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(v) Both speed and altitude were scored between concerned and importance. 

(vi) The student pilots tended to split their dependency on both the ATC and 
CDTI. During spatial information processing tasks. 

(vii) The student pilot showed high concern on the relative position of the intruder 
aircraft. 

(viii) During conflict resolution between intruder and ownership aircraft, the student 
pilot depended on ATC about 65% of the time, rarely used the CDTI, and 
32% of the time, they depended on personal conflict management skills. 

(ix) The student pilots indicated absolute important to altitude availability and 
proximity of the aircrafts, high concerns for distance and vectoring, and little 
worrisome on aircraft type and capabilities. 

(x) During critical decision making such as determining the intruder pilot’s intent 
or closing separation angle, the student pilots depended on CDTI about half 
the time (53.1%) and about one quarter of the time on ATC (28.5%), and 
depended on personal judgments about 18.4% of the time. 

4.1.2. Effect of Weather on Free Flight Aircraft Separation 

4. 1.2.1 Intruder Aircraft Detection Accuracy Under Weather Conditions 

In all weather scenarios, there were remarkable differences between expert and 
student pilots, however the expert pilots showed no statistical differences in the mean 
detection accuracy. The student pilots were difference in detection accuracy, with 
remarkable differences observed in Scenario- 1 under high traffic density and near 
proximity of intruder aircraft. The weather did not show any effect on the type of flying 
tasks, indicating equal mean detection accuracy within the cohort groups either in 
climbing or descending tasks. The detection accuracy was observed to decrease along the 
axis of increasing traffic density and proximity of intruder aircraft. In weather Scenario- 
2, the effect of weather on either traffic density and proximity of aircraft did were not 
apparent and the performance of the experts and student pilots showed some interaction 
across traffic density and proximity. In Scenario-3, detection accuracy was better under 
low traffic density with no statistical differences in task type or proximity of the aircraft. 
In summary, weather scenarios were observed to affect intruder aircraft detection 
accuracies. There was interaction effects between weather Scenario- 1 and Scenario-2 for 
climbing task data generated by both expert- and student-pilots at high traffic density. 
Scenario-3 weather condition provided a poor condition for detection accuracy. This may 
be associated to low visibility. 

4. 1.2.1 Intruder Aircraft Detection Time Under Weather Conditions 

In all weather scenarios, there were remarkable differences between expert and 
student pilots in detection times. Within the cohort group, there were differences in 
detection times due to traffic density. Detection times were better in low traffic density 
and near proximity of the intruder of the aircraft for both expert- and student- pilots. 
Under Scenario- 1 weather, proximity of the intruder aircraft and types of task did not 
show significant effect. In Scenario-2 and Scenario-3 the type of task was not significant. 
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In general, climbing and descending tasks that are mainly maneuvering tasks were not 
affected by the weather conditions. For the student pilots in Scenario-3, the type of task 
showed interaction in a high traffic density. 

4 . 1 . 3 . Effect of Warning on Pilots Decision To Fly Into Weather Condition 
Free Flight 

The following observations were derived in providing the pilots apriori 
information on weather and the subsequent decision to fly into the weather conditions. 

(a) . The experts tended to fly more into adverse weather when pre-wamed at a distance 
“far” away from the weather location. 

(b) , The experts tended to avoid flying into adverse weather even if pre-warning and 
reminder were provided when the proximity was close to weather conditions. 

(c ). The student pilots practiced the same behavioral decision as the expert pdots, except 
that they were more cautious, flying more into weather Scenario- and less in more 
adverse weather scenarios. 

We encounter behaviors that were some how inconsistent. For example, the 
expert pilots showed more flight into weather Scenario- 1 and less in Scenario-2 under 
pre- warning condition, and fly more into Scenario-2 than Scenario- 1 under no-warning 
condition. These decisions remain elusive for any useful explanation. One explanation, 
however, may be attributed to ability to estimate risk under the flight conditions ( O’Hare 
& Smitheram, 1995). In general, when pilots were warned of the weather conditions, they 
were more likely to fly their aircraft into it, but mostly when the warning was not close to 
the weather location. This result is somewhat consistent with the finding by Goh & 
Weigmann (2001a), Peterson & Uhlarik (1999), and Orasanu, Martin, & Davison (2001). 

4.2 Summary 

Overall, the results obtained from the behavioral analysis showed that in general, the 
ability to recognize the conflict incidents, followed by the ability to acquire the spatial 
location of the intruder aircraft relative to ownership aircraft were judged to be the major 
cognitive tasks as perceived by the participants during self-separation. Further, the 
participants rarely used CDTI during conflict management related to aircraft separation, 
but used CDTI highly during decision-making tasks. 

In all weather scenarios, there were remarkable differences between expert and 
student pilots in detection times. In summary, weather scenarios were observed to affect 
intruder aircraft accuracies. There was interaction effects between weather Scenario- 1 
and Scenario-2 for climbing task data generated by both expert- and student-pilots at high 
traffic density. Scenario-3 weather condition provided a poor condition for detection 
accuracy as well as detection time increase. This may be associated to low visibility. In 
general, intruder aircraft detection times were not affected by the weather conditions 
during climbing and descending tasks. 

The decision of pilots to fly into the weather condition was dependent in part on the 
warning distance to the location of the weather. In general, when pilots were warned of 
the weather conditions, they were more likely to fly their aircraft into it, but mostly when 
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the warning was not close to the weather location. There are many factors that may 
contribute to the behavior of the pilots to fly into weather conditions. These are the 
possibilities: 

(a) With sufficient distance they can plan and develop coping strategies to 
deal with the weather. 

(b) They may have forgotten the warning and depend only on the current 
expected state of information (Johnson & Tversky, 1984). 

(c) It may be due to an intent to save cost, fuel economy, and/or arrival time 
to destination. This is attributed to the so-called “sunk-cost effect 
(Wiegmann, Goh, & O’Hare, 2002). 

(d) It may be attributed to risk taking behavior, in which case, the pilot group 
is said to be risk prone (Edwards, 1987). 

While the current research provides important findings in effects of VFR flight into 
IMC, several tasks and unanswered questions remain. Therefore, additional research 
is needed to provide answers to such questions, such as, 

(i) Whether intennittent weather warning will affect the decision of pilots to fly 
into weather. If so, what should be the optimal warning schedule and distance 
to weather location. 

(ii) Why pilots who receive adverse warning closer to destination will divert or 
continue into the weather. 

(iii) Why pilots depend less on ATC or CDTI in managing separation conflicts. 

(iv) Why during critical decision making such as determining the intruder pilot’s 
intent or closing separation angle, the experts tended to rely almost on their 
mental models and CDTI in decision-making behaviors. If so, how can CDTI 
be designed to capture the pilot’s mental model and reduce cognitive 
workload? 

It should be noted that the behavioral approach used to uncover the pilot’s perception 
of separation tasks is not complete and less detailed. It is suggested that cognitive task 
analysis opined on behavior principles be used to study and develop more comprehensive 
model of pilot’s behavior during separation at different weather and flying task scenarios. 
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Appendix: Self-Report Questionnaire to Determine Pilot’s Information Acquisition 
During Aircraft Separation Task. 


Please answer the following questions as they are relevant to the all aspects of the flying 
and self-separation tasks you have just completed. List the resources the space provided 
according to the order of priority, with the highest priority first. The resources are Air 
Traffic Controller (ATC), Cockpit Display Traffic Information (CDTI), and self 
knowledge (SELF). 

Question 1: Where do you get most of the information during the processing the 
following cognitive tasks in: 

1 . Determining aircraft converging headings: 

2. Determining aircraft speed differences: 

3. Determining the relative spatial position of the intruder aircraft: 

4. Determining the minimum lateral separation: 

5. Determining the minimum longitudinal separation: 

6. Determining the minimum separation height: 

7. Determining the intruder pilot intent: 

8. Negotiating for airspace right of way: 

9. Deciding to divert your aircraft from the intruder: 

10. Determining dead reckoning situation: 

1 1 . Closing the separation angles from the intruder: 

12. Determining the vectoring space available: 

13. Determining the available altitudes for maneuvering: 

14. Determining the intruder aircraft characteristics: 

15. Determining the proximity of your aircraft to that of the intruder: 

16. Determining your aircraft distant from the intruder: 

17. Determining your aircraft time to collision from the intruder: 

Question 2: During the aircraft self-separation, rate the following cognitive tasks by 
its importance (3), relevance (2) , or concern (1) as it pertains to task performance: 

18. Recognition of intruder conflict: 

19. Knowledge of intruding aircraft: 

20. Conflict reconciliation between intruder aircraft pilot and you: 

21 . Knowledge of intruder pilot intent: 
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